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Period for Reply 



A SHORTENED STATUTORY PERIOD FOR REPLY IS SET TO EXPIRE 3 MONTH(S) FROM 
THE MAILING DATE OF THIS COMMUNICATION. 

- Extensions of time may be available under the provisions of 37 CFR 1 .136(a). In no event, however, may a reply be timely filed 
after SIX (6) MONTHS from the mailing date of this communication. 

- If the period for reply specified above is less than thirty (30) days, a reply within the statutory minimum of thirty (30) days will be considered timely. 

- If NO period for reply is specified above, the maximum statutory period will apply and will expire SIX (6) MONTHS from the mailing date of this communication. 

- Failure to reply within the set or extended period for reply will, by statute, cause the application to become ABANDONED (35 U.S.C. § 133). 
Any reply received by the Office later than three months after the mailing date of this communication, even if timely filed, may reduce any 
earned patent term adjustment. See 37 CFR 1.704(b). 

Status 

1 )S Responsive to communication(s) filed on 02 February 2004 . 
2a)^ This action is FINAL. 2b)0 This action is non-final. 

3) D Since this application is in condition for allowance except for formal matters, prosecution as to the merits is 

closed in accordance with the practice under Ex parte Quay/e, 1935 CD. 11, 453 O.G. 213. 

Disposition of Claims 

4) ^ Claim(s) 1^9 is/are pending in the application. 

4a) Of the above claim(s) is/are withdrawn from consideration. 

5) Q Claim(s) is/are allowed. 

6) ^ Claim(s) 1^9 is/are rejected. 

7) D Claim(s) is/are objected to. 

8) D Claim(s) are subject to restriction and/or election requirement. 

Application Papers 

9) D The specification is objected to by the Examiner. 

10) D The drawing(s) filed on is/are: a)Q accepted or b)D objected to by the Examiner. 

Applicant may not request that any objection to the drawing(s) be held in abeyance. See 37 CFR 1 .85(a). 
Replacement drawing sheet(s) including the correction is required if the drawing(s) is objected to. See 37 CFR 1.121(d). 

1 1) D The oath or declaration is objected to by the Examiner. Note the attached Office Action or form PTO-152. 

Priority under 35 U.S.C. § 119 

12) 0 Acknowledgment is made of a claim for foreign priority under 35 U.S.C. § 1 19(a)-(d) or (f). 
a)D All b)D Some * c)Q None of: 

1 0 Certified copies of the priority documents have been received. 

2.0 Certified copies of the priority documents have been received in Application No. . 

3.0 Copies of the certified copies of the priority documents have been received in this National Stage 
application from the International Bureau (PCT Rule 17.2(a)). 
* See the attached detailed Office action for a list of the certified copies not received. 
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DETAILED ACTION 
Response to Amendment 

1. Applicants, amendment filed February 2, 2004, has been entered and made of 
record. 

2. Applicants are respectfully requested to resubmit a certified copy of the 
application 199 55 919.8 filed in Germany. 

3. Applicants are respectfully requested to provide support for the simultaneous 
amendment of the same subject matters in Claim 1 and Page 32, Lines 2-22 of the 
specification 

4. Applicants' arguments have been fully considered but they are not persuasive. 
Murray et al.'s teachings is not limited to recognizing a single object (Column 1, Lines 
52-55). Likewise, Lawrence et al.'s invention is not limited to recognizing a single object 
that has a set of features, by sampling a plurality of local regions of an image. On the 
contrary, Lawrence et al. deal with different objects on the face such as nose, mouth, 
etc.(Column 1 , Lines 6-8, and 44-57). Prior art of record disclose multiple object 
recognition based on the following methodology that meet the recited steps (a) to (f): 

As it is known in the art of object recognition, conventionally all pixels of received 
images are utilized for classification purposes. As an standard procedure, foreground 
pixels are used to form a reduced image, and background pixels or irrelevant pixels are 
ignored. 

The reduced images (homogeneous regions after image segmentation) are 
filtered to form at least filtered images (conventional preprocessing). 
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Murray et al. disclose the filtered images are further classified by a group of 
different classifiers (Classifiers 10, 1 1 , 22 and 23), whereby each classifier works on the 
basis of learned rules and based on a respective characterizing vector to form classified 
images. 

Murray et al. further disclose the classified images are merged (Assessor unit 12) 
and a decision is made, based on the merged images, whether any of all remaining 
pixels are still relevant and if so to which different object class the still relevant pixel 
belongs. 

Claim Rejections - 35 USC § 103 

5. The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set 
forth in section 102 of this title, if the differences between the subject matter sought to be patented and 
the prior art are such that the subject matter as a whole would have been obvious at the time the 
invention was made to a person having ordinary skill in the art to which said subject matter pertains. 
Patentability shall not be negatived by the manner in which the invention was made. 

6. Claims 1- 3, 6, 9 are rejected under 35 U. S. C. 103 (a) as being unpatentable 
over Murray et al., (hereinafter Murray), US 6597800, and Lawrence et al., (hereinafter 
Lawrence), US 6038337. 

As per claim 1 , Murray teaches a method for multiple object recognition (Column 
1 , Lines 52-56) on an image pixel plane of received images comprising the following 
steps: 

(a) roughly classifying (10) all pixel points of said received images whether or not 
a pixel point is relevant for said multiple object recognition to provide relevant (part of 
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the object) pixel points and eliminate irrelevant pixel points (Column 1 , Lines 47-56; 
Column 4, Lines 22-43); 

(b) forming a reduced image (11) based on said relevant pixel points as roughly 
classified in step (a) (Binarization; Column 4, Lines 27-30); 

(c) filtering (20) each reduced image (1 1 ) for forming at least two corresponding 
decomposed or filtered images (21, 22, 23) (binary mask, Column 4,Lines 30-33 and 
Sobel filter, Column 4, Lines 37-40) whereby image components relevant for said 
multiple object recognition (object pixels (binary mask), object boundary (Sobel filter)) 
are retained in said at least two filtered images (Filtering homogeneous regions 
simultaneously filters all objects of interest that will generate a plurality of filtered 
images); 

(d) further classifying (30) said at least two filtered images for providing classified 
images, wherein said further classifying is performed by a group of different classifiers 
(Classifier 10 and 22, Fig. 1) which operate in accordance with learned rules to allocate 
said classified images to different object classes, wherein each different classifier works 
independently of any other classifier of said group of different classifiers (Figure 1 ; 
Column 1, Lines 57-67, Column 2, Lines 40). 

Murray does not specifically teach that his classifiers use a characterizing vector, 
however, Lawrence teaches: 

wherein each of said classifiers operates based on a characterizing vector 
forming an input information for its respective classifier ( Column 4, Lines 6-15); 

Murray goes on to teach: 
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(e) merging (40) said classified images in accordance with an algorithm (Figure 
1; Column 6, Lines 8-15) to perform a combined global evaluation for each class of said 
different object classes, for providing merged images (41 A, 41 B, 41C) (assessor unit, 
Column 5, Line 63 through Column6, Line 15, see Fig. 1 ; Also teachings of Lawrence et 
al. concerning providing images of different portions of the face such as nose, mouth, 
etc. ); and 

(f) deciding (50), on the basis of said merged images, whether a pixel point of all 
remaining pixel points is still relevant and if so to which of said different object classes 
(C1 or CZ) each relevant pixel point belongs (Column 5, Lines 56-67, Column 6, Lines 
1-18; Also the inherent characteristic of Lawrence et al. backpropagation neural 
network). 

It would have been obvious to one of ordinary skill in the art to use the feature 
vector of Lawrence in the general feature extraction system of Murray to overcome the 
limitations of the prior art with regard to variations in scale, orientation, and illumination 
of the target image data versus the reference image data. 

As per claim 2, Murray teaches: 

providing a set of predetermined first criteria (greater homogeneity, Column 3, 
Lines 4748) for performing said rough classifying step (a), and providing a second set of 
second predetermined (change of illumination intensity, Column 3, Lines 35-36) criteria 
for performing said filtering step (c). 

As per claim 3, Murray does not teach forming characterizing vectors, however, 
Lawrence teaches: 
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further comprising acquiring vicinity (pixels around center pixel) image data 
representing a vicinity of a respective relevant pixel point of a corresponding filtered 
image, and forming said characterizing vector from said vicinity image data (Column 4, 
Lines 23-28). It would have been obvious to one of ordinary skill in the art to use the 
vicinity vectors of Lawrence in the feature extraction system of Murray because the 
vicinity vector data is partially invariant to variations in intensity of the image. 

As per claim 6, Murray or Lawrence teach: 

comprising using, as said group of different classifiers, a neural network (Murray, 
multi layer perceptron technique, Column 5, Lines 35-36, or Lawrence, figure 2, element 
260, Column 3,1. 2, "self-organizing map neural network") capable of learning 
(Lawrence, Column 4, Lines 31-36, with regard to Murray it is well-known that multi 
layer perceptron can be trained) for performing said step of further classifying (30). 

As per claim 9, Murray teaches: 

representing recognized objects by pixel clusters (clustering is used in K-Nearest 
Neighbor classification, Column 5, Lines 35-36) in an image (51) that represents a 
decision regarding said combined global evaluation while performing said deciding step 
(50) (assessor unit, Column 6, Lines 15-20, output 24, figure 1). 
7. Claims 4 and 7 are rejected under 35 U.S.C. 103(a) as being unpatentable over 
Murray and Lawrence as applied to claims 1 above, and further in view of Hutchenson 
et al., (hereinafter Hutchenson), US 5,465,308. 

As per claim 4, Murray does not teach weighting. But Lawrence teaches that the 
a weighting factor can be added to each pixel to compensate for some invariance prior 
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to being sent to the neural network, but does not specifically address a weighting 
system for different classes. However, Hutchenson teaches: 

comprising providing different weighting factors or evaluation numbers 
representing different classes of objects to be recognized (Rank vectors, Column 17, 
Lines 4655), and assigning or allocating certain weighting factors or evaluation numbers 
of said different weighting factors to each relevant pixel point (each feature component 
(e.g., pixel)) is thereby marking each relevant pixel point (as taught by Lawrence, 
Column 4, Lines 26-30) with regard to which of said different classes of objects the 
marked pixel point belongs. 

As per claim 7, neither Murray nor Lawrence teach using the characterizing 
vectors to create rules for the neural network, however, Hutchenson teaches: 

comprising selecting from relevant pixel points of said filtered images (21, 22, 23) 
characterizing or feature vectors representing features of said relevant pixel points of 
said rough classifying (10) (input feature vectors, Column 18, Lines 22-25), and forming 
rules 

(changing weights, Column 18, Lines 26-30) for said neural network from said 
characterizing or feature vectors. 

8. Claim 5 is rejected under 35 U.S.C. 103(a) as being unpatentable over Murray 
and Lawrence as applied to claims 1 above, and further in view of Knecht et al., 
(hereinafter Knecht), US 4,881 ,270. 

As per claim 5, neither Murray nor Lawrence teach the limitations of claim 5, 
however, Knecht teaches: 
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sorting said vicinity image data in a spiral pattern into a vector of features 
(although Knecht teaches using rows and columns (Column 5,Lines 1 1-20) this is 
merely a design choice on how to present the data in vector form) into a vector of 
coefficients, applying a rapid (fast) Fourier transform (Column 5, Lines 34-40) to said 
vector of coefficients to form transformation coefficients (Column 5, Lines 52-54), and 
forming said characterizing vectors of an absolute value of said transform coefficients 
(Column 5, 1. 7-Column 6, 1. 7). 

It would have been obvious to one of ordinary skill in the art to use the absolute 
values of Fourier transform coefficients as taught by Knecht in the system of Murray and 

Lawrence because the well known Fourier power spectrum of the frequencies 
present in the image data axe easily manipulated as shown in Knecht's figure 5. 
9. Claim 8 is rejected under 35 U.S.C. 103(a) as being unpatentable over Murray 
and Lawrence as applied to claims 1 above, and further in view of Wang et al. 
(hereinafter Wang), "Automatic Target recognition Using a Feature-Decomposition and 
Data Decomposition Modular Neural Network". 

As per claim 8, neither Murray nor Lawrence specifically teach using a statistical 

process for global evaluation, however, Wang teaches: 

further comprising performing said merging step in accordance with a statistical 
process (fig. 8, summation for mixture of experts, p. 1 18, equations 16 and 17) for 
obtaining said global evaluation (final classification, p. 1118, Column 2 , and further 
comprising using in said statistical process information based at least on one of a type 
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(page 1119, Column 2, Lines 5-11), position and content of said received images (1) to 
be evaluated. 

Conclusion 

10. Applicant's amendment necessitated the new ground(s) of rejection presented in 
this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP 

§ 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 
CFR 1.136(a). 

A shortened statutory period for reply to this final action is set to expire THREE 
MONTHS from the mailing date of this action. In the event a first reply is filed within 
TWO MONTHS of the mailing date of this final action and the advisory action is not 
mailed until after the end of the THREE-MONTH shortened statutory period, then the 
shortened statutory period will expire on the date the advisory action is mailed, and any 
extension fee pursuant to 37 CFR 1 .136(a) will be calculated from the mailing date of 
the advisory action. In no event, however, will the statutory period for reply expire later 
than SIX MONTHS from the date of this final action. 

Contact Information 

1 1 . Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Mehrdad Dastouri whose telephone number is (703) 
305-2438. The examiner can normally be reached on Monday to Friday from 8:00 a.m. 
to 4:30 p.m.. 
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If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Amelia Au can be reached on (703) 308-6604. The fax phone number for 
the organization where this application or proceeding is assigned is 703-872-9306. 

Information regarding the status of an application may be obtained from the 
Patent Application Information Retrieval (PAIR) system. Status information for 
published applications may be obtained from either Private PAIR or Public PAIR. 
Status information for unpublished applications is available through Private PAIR only. 
For more information about the PAIR system, see http://pair-direct.uspto.gov. Should 
you have questions on access to the Private PAIR system, contact the Electronic 
Business Center (EBC) at 866-217-9197 (toll-free). 




Mehrdad Dastouri 
Primary Examiner 
Art Unit 2623 
April 18, 2004 



